Cross device image recognition improvement

ABSTRACT

A method includes modifying values of image data of a first image of an object, the first image taken by a user using a user equipment. The modifying is performed to map one or more color characteristics of one or more color components of the user equipment to corresponding one or more color characteristics for the one or more color components of a reference device. The modifying creates a modified image. On the user equipment, comparisons are performed between the modified image and a number of second images of objects taken by the reference device. Apparatus and program products are also disclosed. Additionally, a user interface is disclosed that provides for training a device model that is used during the mapping.

TECHNICAL FIELD

This invention relates generally to image capture and, morespecifically, relates to image matching using different devices.

BACKGROUND

In a visual search, a user points a camera of a device such as a phoneat an object and has the phone recognize the object using an imagecaptured by the camera. Once the object is recognized, the phone cantake a multitude of actions. For example, the object might be abuilding, and the phone could present the user with additionalinformation such as the name, address, occupants, or architecture of thebuilding, or search results, e.g., from the Internet, pertaining to thebuilding. Illustratively, the results could be of restaurants in or nearthe building. As another example, the object could be a poster for amovie and the phone could present information about the movie, a trailerfor the movie, local theaters showing the movie, and the like. As yetanother example, the object could be a barcode for a product, and thephone could return a detailed description of the product, nearby storeshaving the product, and the prices of the product at those stores.

In order to recognize an object, the phone can access a visual searchdatabase, typically on the phone. The visual search database may also beat a remote server. The images in the visual search database arecommonly called tags. These images are either captured with a dedicateddevice by a professional service team, or are provided by multiplesources using a variety of cameras/devices.

For image matching, the captured image is saved in the memory. Then thisinput image is matched against the images in the visual search database.Typically, the phone will present multiple possible matches to a userfor confirmation as to which of the images (if any) is a match to theobject the user captured with his or her image. When an object in apresented image matches the object in the user-taken image, informationassociated with the object will be displayed. Typically, the visualsearch database contains images of all possible objects the user mayneed to recognize.

This type of visual search has many benefits, but could be improved.

SUMMARY

In an exemplary embodiment, a method includes modifying values of imagedata of a first image of an object, the first image taken by a userusing a user equipment. The modifying is performed to map one or morecolor characteristics of one or more color components of the userequipment to corresponding one or more color characteristics for the oneor more color components of a reference device. The modifying creates amodified image. On the user equipment, comparisons are performed betweenthe modified image and a number of second images of objects taken by thereference device.

In another exemplary embodiment, an apparatus is disclosed that includesone or more processors configured to cause the apparatus to perform atleast modifying values of image data of a first image of an object, thefirst image taken by a user using the apparatus. The modifying isperformed to map one or more color characteristics of one or more colorcomponents of the apparatus to corresponding one or more colorcharacteristics for one or more color components of a reference device.The modifying creates a modified image. The one or more processors areconfigured to cause the apparatus to perform performing comparisonsbetween the modified image and a number of second images of objectstaken by the reference device.

In an additional exemplary embodiment, a computer program product isdisclosed that includes a computer-readable storage medium bearingcomputer program code embodied therein for use with an apparatus. Thecomputer program code includes code for modifying values of image dataof a first image of an object, the first image taken by a user using theapparatus. The modifying is performed to map one or more colorcharacteristics of one or more color components of the apparatus tocorresponding one or more color characteristics for the one or morecolor components of a reference device. The modifying creates a modifiedimage. The code also includes code for performing comparisons betweenthe modified image and a number of second images of objects taken by thereference device.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other aspects of embodiments of this invention aremade more evident in the following Detailed Description of ExemplaryEmbodiments, when read in conjunction with the attached Drawing Figures,wherein:

FIG. 1A shows a simplified block diagram of an exemplary user equipmentsuitable for use in practicing the exemplary embodiments of thisinvention;

FIG. 1B shows a more particularized block diagram of a user equipmentsuch as that shown at FIG. 1A;

FIG. 1C shows a more particularized block diagram of a memory ormemories shown also in FIG. 1B;

FIG. 2 is a block diagram of actions taken in an exemplary embodiment ofthe invention, and includes an example of a user interface used fortraining;

FIG. 3 is an illustration of using a device model to apply differenttransforms to color components;

FIG. 4 is a block diagram of actions taken for white balancing;

FIG. 5A is an exemplary histogram of a color component for a referencedevice;

FIG. 5B is an exemplary histogram of a color component for a Device A(e.g., a user equipment used by a user);

FIG. 5C is an exemplary histogram of differences between the histogramsshown in FIGS. 5A and 5B;

FIG. 6 is an illustration of using threshold points and true dynamicrange to segment the color components and calculate color transforms fordifferent devices; and

FIG. 7 illustrates a pipeline for determining true dynamic range of animage relative to a user equipment.

DETAILED DESCRIPTION OF THE DRAWINGS

As stated above, visual search has many benefits but could be improved.In particular, the images (commonly called “tags”) in the visual searchdatabase may be captured by multiple devices, and it is likely thedevice used for recognition is not the same device used for capturingthe tags. A frequent problem is that because the images are captured bydifferent devices, the accuracy of search results will deteriorate. Themismatch of images coming from different devices accounts for a largeportion of total mismatches, and is one of the main factors in theoverall accuracy of the visual search system. The images provided bydifferent devices are more difficult to match, because, e.g., thedifferences in sensor characteristics between devices and thedifferences in the internal image processing pipeline in each device.The internal image pipeline processes the raw images in different ways,before generating the processed images that are then stored. Imagefeature mismatch is different depending on devices. There is no singleimage processing pipeline that can compensate for all the differentdevices.

These issues reduce the image recognition accuracy and have a stronginfluence on the user experience using a mobile visual search system.Existing attempts at correcting these problems include using dedicatedtags for more reliable recognition. This approach uses the same devicefor tagging and recognition, but a drawback is the process is notscalable. This is particularly true considering there are a wide varietyof cameras/devices to support, which would require a very largedatabase. Another attempt includes attempting to create single genericimage matching algorithm that covers all devices with optimalperformance. However, such algorithms have previously not beensuccessful. Further, there is no training option on devices, meaningthat there previously was no way to compensate image processing forindividual devices.

This invention addresses these issues by using, in certain exemplaryembodiments, device-dependent white balance, adaptive dynamic rangeadjustment across devices, and on-device interface design. The dynamicrange adjustment affects saturation. The term “adaptive” means theadjustment also depends on which segment (e.g., range) of a histogram apixel intensity level falls in, and the adjustment therefore may haveinfluence on contrast. This invention greatly improves the imagerecognition accuracy across devices. It also provides a mechanism forusers to enable an image processing algorithm on their devices.

Before proceeding with additional description of the invention,attention is first directed to an exemplary user equipment 10 that issuitable for carrying out exemplary embodiments of the invention. FIG.1, including FIG. 1A and FIG. 1B and FIG. 1C, illustrates detail of anexemplary user equipment 10 in both plan view (FIG. 1A) and sectionalview (FIG. 1B), and a more detailed section view of a memory or memories(FIG. 1C). It is noted that the user equipment 10 may also be referredto as a mobile device or a device herein. The invention may be embodiedin one or some combination of the function specific components shown inFIG. 1. At FIG. 1A, the user equipment 10 has a graphical displayinterface 20 and a user interface 22 illustrated as a keypad butunderstood as also encompassing touch screen technology at the graphicaldisplay interface 20 and voice recognition technology received at themicrophone 24. A power actuator 26 controls the device being turned onand off by the user. The exemplary UE 10 has a camera 28 which is shownas being forward facing (e.g., for video calls) but may alternatively oradditionally be rearward facing (e.g., for capturing images and videofor local storage). The camera 28 is controlled by a shutter actuator 30and optionally by a zoom actuator 32 which may alternatively function asa volume adjustment for the speaker(s) 34 when the camera 28 is not inan active mode.

Within the sectional views of FIGS. 1B and 1C are seen the multipletransmit antennas and possibly also multiple receive antennas 36 thatare typically used for radio frequency (e.g., cellular) communication.The antennas 36 may be multi-band for use with other radios in the userequipment 10. The operable ground plane for the antennas 36 spans, in anexemplary embodiment, the entire space enclosed by the housing 75 thoughin some embodiments the ground plane may be limited to a smaller area,such as disposed on a printed wiring board on which the power integratedcircuit 38 is formed. The power integrated circuit 38 controls poweramplification on the channels being transmitted and/or across theantennas 36 that transmit simultaneously where spatial diversity isused, and amplifies the received signals. The power integrated circuit38 outputs the amplified received signal to the radio frequency (RF)integrated circuit 40 which demodulates and downconverts the signal forbaseband processing. The baseband (BB) integrated circuit 42 detects thesignal which is then converted to a bit stream and finally decoded.Similar processing occurs in reverse for signals generated in the userequipment 10 and transmitted from it.

Those signals that go to and from the camera 28 pass through animage/video processor 44 (e.g., part of an image pipeline) thatprocesses the image frames 111 (stored as images 120 after processing).A separate audio processor 46 may also be present controlling signals toand from the speakers 34 and the microphone 24. The graphical displayinterface 20 is refreshed from a frame memory 48 as controlled by a userinterface integrated circuit 50, which may process signals to and fromthe display interface 20 and/or additionally process user inputs fromthe keypad 22 and elsewhere.

Certain embodiments of the user equipment 10 may also include one ormore secondary radios such as a wireless local area network (WLAN) radio37 and a BLUETOOTH (BT) radio 39, which may incorporate an antenna onthe integrated circuit or be coupled to an antenna off the integratedcircuit. As is known, BLUETOOTH is a wireless protocol for exchangingdata over short distances. Throughout the user equipment 10 are variousmemory/memories 100 such as random access memory (RAM) 43, read onlymemory (ROM) 45, and in some embodiments removable memory such as theillustrated memory card 47. On the memories 100, various programs 110may be stored. The programs 110 include, e.g., an operating system and aprogram for carrying out the exemplary operations described herein. Allof these components within the user equipment 10 are normally powered bya portable power supply such as a battery 49.

If the integrated circuits 38, 40, 42, 44, 46, 50 are embodied asseparate entities in a user equipment 10, these may operate in a slaverelationship to the main processor 72 (also an integrated circuit),which may then be in a master relationship to them. Embodiments of thisinvention may be disposed across various integrated circuits andmemories as shown, or disposed within another processor that combinessome of the functions described above for FIGS. 1B and 1C. Any or all ofthese various processors of FIG. 1B access one or more of the variousmemories of FIGS. 1B and 1C, which may be on integrated circuit with theprocessor or separate from the processor.

Note that the various processors (e.g., 38, 40, 42, 44, 46, 50, 72) thatwere described above may be combined into a fewer number than describedand, in a most compact case, may all be embodied physically within asingle integrated circuit. An integrated circuit, as is known, is anelectronic circuit built on a semiconductor (or insulator) substrate,usually one of single-crystal silicon. The integrated circuit, oftencalled a chip, may be packaged in a hermetically sealed case or anon-hermetically sealed plastic capsule, with leads extending from itfor input, output, and power-supply connections, and for otherconnections that may be necessary when the device is put to use. It isalso noted that any of the processors (e.g., 38, 40, 42, 44, 46, 50, 72)may also include other circuitry, such as discrete circuitry, and mayinclude such circuitry as programmable logic devices and gate arrays.The program(s) 110 may therefore be implemented as hardware elements, oras software that executes on one or more of the processors (e.g., 38,40, 42, 44, 46, 50, 72), or as some combination of hardware elements andsoftware.

In the example of FIG. 1, the image(s) 120 are image frames 111 taken bycamera 28 and after processing by image/video processor 44. The image(s)120 may or may not be stored in a compressed form. The image(s) 120 areof a corresponding object(s) 121 and are to be compared with the tags135-1 through 135-N in the tag database (DB) 130. For clarity, theimages in the database 130 are called tags herein, to distinguish theimages in the database 130 from images taken by the user of userequipment 10. Thus, the database 130 is a tag database having tags 135.Each tag 135 is an image of an object 136. The tag database 130typically resides on the user equipment 10, but may also reside on,e.g., a server 140, connectable via wired or wireless network(s) 150. Inthe example of FIG. 1B, the network 150 is a wireless networkconnectable to the user equipment 10 through wireless links 151, 152.

In an exemplary embodiment, the program 110 has program code thatmodifies values of image data in image 120 to create a modified image125. The adjustment is made to map a color characteristic of the colorcomponent (such as red, green blue components) of the user equipment toa corresponding color characteristic of a reference device. The colorcharacteristics and color components of the reference device are known.The modified image 125 is then compared with tags 135 in the tagdatabase 130, with the ultimate goal being to match object 121 with anobject 136. In certain exemplary embodiments, the color characteristiccould be one or more of white balance, contrast, or saturation, althoughother color characteristics may be used.

Referring now to FIG. 2 (with appropriate reference to FIG. 1), a blockdiagram is shown of actions taken in an exemplary embodiment of theinvention. This block diagram includes an example of a user interfaceused for training. In the example of FIG. 2, it will be assumed thatmemories 100 (see FIG. 1) have computer program code (e.g., program110), and the memories 100 and the computer program code are configured,with the processor 72 (and/or other processors shown in FIG. 1), tocause user equipment 10 to perform the actions shown in the followingfigures. For simplicity, it is assumed that a processor is configured toperform these actions.

In block 205, a user takes an image 120 of an object. In block 210, theprocessor is configured to cause a comparison between the user-takenimage 120 and tags 135. In block 208, a user equipment 10 is shown beingoperated by a user. The user is using the user interface 22. Part ofuser interface 22 includes the displayed interface 201, which shows aconfirmation dialog 202 and two tags 204, 206. The confirmation dialog202 asks the user to confirm the recognition result, as embodied in oneof the two tags 204 and 206. The user would select the appropriate tag204, 206, then highlight and select the confirmation dialog 202. Notshown is that the user might not confirm the recognition result, and,e.g., be presented with another two tags 204, 206. It is noted that therecognition result means the object 121 matches an object 136.

In block 215, a device model is estimated. The device model isdetermined such that applying the device model to image data from animage 120 adjusts one or more color characteristics of one or more colorcomponents of an image 120 to map the one or more color components tocorresponding color characteristics for those color components of areference device. Block 215 is described in more detail below, but caninclude white balance adjustment (block 217), and adaptive dynamic rangeadjustment (block 218). Each of blocks 217 and 218 defines part of atransformation (described in more detail below). Block 215 uses theuser-taken image 120 and the selected tag 204 (or tag 206), along withknown color characteristics about the device that took the tag 204 (or206), as is described in more detail below.

In an exemplary embodiment, the user is prompted via a dialog 218 on thedisplayed interface 201 to use the device model determined in block 215when processing future images 120. It is assumed in the followingdescription that the user responds affirmatively and therefore blocks220, 225, and 230 are performed (blocks 220, 225, and 230 would not beperformed if the user responds negatively in response to the dialog218). It is noted that the text in dialog 218 and the location of dialog218 in the blocks of FIG. 2 are merely exemplary. For instance, thedialog 218 could be performed before block 215, and ask the user if amodel should be created based on the confirmed result (e.g., and used inthe future for other images 120). If the user responds negatively, block215 (and subsequent blocks) would not be performed; if the user respondsaffirmatively, blocks 215, 220, 225, and 230 would be performed.

In block 220, the device model is applied to a new image 120 to create amodified image 125. In block 225, the modified image 125 is comparedwith tags 135 in tag database 130 to determine tags 231, 233corresponding to objects 136 that are possible matches to the object 121in the image 120. The tags 231, 233 are presented to the user in block230, and the previous confirmation dialog 202 would also be shown indisplayed interface 201. In this example, reference 238 illustrates thatimproved image recognition should occur after image correction through adevice model. That is, the tags 231 and 233 should be closer to theimage 120 and fewer (or no) iterations of presenting different tags 231and 233 to the user should be performed, relative to if the device modelis not applied to an image 120.

Thus, FIG. 2 shows a feature for a new interface (e.g., user interface22 including displayed interface 201) to provide device-specific imageprocessing for optimal image recognition. As shown, users are promptedto confirm the images they just captured. The images are then comparedwith the tags for the same objects in the visual search database (e.g.,tag database 130). A device model is obtained and saved internally. Theuser can then choose to let the system process the images optimallyaccording to the device model in the future.

FIG. 3 is an illustration of using a device model to apply differenttransforms to color components. In this example, a device model 310 isapplied to color components 330-1, 330-2, and 330-3. Device A 320 hasthree different color components 330 in this example. The colorcomponents are typically red, green, and blue. In the example of FIG. 3,color component 330-1 corresponds to the color red, color component330-2 corresponds to the color green, and color component 330-3corresponds to the color blue. Each pixel (e.g., in camera 28) is atriad of red, green, blue. Each of the red, green, or blue is generallyassigned a number of levels, such as eight bits or 256 levels (from zeroto 255). Zero indicates no color and 255 indicates maximum color. It isnoted that these values and colors are merely exemplary.

The device model 310 includes a transform 315-1, 315-2, and 315-3corresponding to each color component 330-1, 330-2, and 330-3,respectively. The Device B, which is typically a known reference device,also has color components 360-1, 360-2, and 360-3 that correspond tocolor components 330-1, 330-2, and 330-3. Thus, the “input” colorcomponents 330 are transformed to be the “output” color components 360.In an exemplary embodiment, the device model 310 is able to stretch orcompress intervals (e.g., segments) of individual color components 330(see FIG. 4 and associated text). This creates a white balancing thatcan vary according to pixel density values.

It is noted that each color component and the color characteristics(such as white balance, contrast, and/or saturation) define a colorspace for a device. In one sense, the device model 310 maps from thecolor space 320 of Device A to the color space of Device B 350 (usually,a reference device having known color components and known colorcharacteristics). The color components and associated colorcharacteristics are defined by the device, e.g., by the camera 28 and bywhatever video processing is performed (e.g., by the image/videoprocessor 44).

FIG. 3 also shows an image 120 having image data 390 that is separatedinto components 391-1, 391-2, and 391-3 (in this case, red, green, andblue components, respectively). The values corresponding to thesecomponents 391 are transformed via transforms 315 into modified values395-1, 395-2, and 395-3. The modified values 395 have colorcharacteristics (e.g., white balance, contrast, and/or saturation) thatare similar after mapping through a transform 315 to the colorcharacteristics of the reference device (Device B in this example). Thetransforms 315 map the color characteristics of the color components 330to the color characteristics of the color components 360. It is notedthat the image data 390 may be pixel data (e.g., three values for eachpixel in an image 120), compressed data, or other types of image data.

Referring to FIG. 4 (with appropriate reference to preceding figures), ablock diagram is shown of actions taken for white balancing. In block410, histograms are computed of color components on Device A (the userequipment 10 being used by a user for image comparison) and thereference device (the device that takes the images stored as tags 135).To determine the histograms, divide the dynamic range evenly intointervals (e.g., bins) and then count the number of pixels withintensity belonging to each bin. This is shown in FIG. 5A, which is anexemplary histogram of a color component for a reference device, and inFIG. 5B, which is an exemplary histogram of a color component for aDevice A. FIGS. 5A and 5B (and 5C) are shown as four bins for ease ofdescription, but typically there would be more bins.

In block 420, only (in an exemplary embodiment) the color componentswith histogram distortion exceeding a preset threshold are transformed,and the color components that do not exceed a present threshold areignored. FIG. 5C is an exemplary histogram of differences between thehistograms shown in FIGS. 5A and 5B. Bin number two in FIG. 5C has adifference value above a threshold. More specifically, the overalldifference (for example, the sum of differences in each bin) will becompared to a threshold; if the overall difference is large enough, theentire color component will be transformed. It is noted that the binswith large differences have no direct relationship to the thresholdpoints in FIG. 6. Typically, there are many fewer threshold points thenthe number of bins with large discrepancy. The threshold points areestimated so that the piece-wise transforms will optimally compensatefor the histogram differences.

In block 430, a piece-wise linear mapping of the color components beingtransformed is estimated from Device A to the reference device. In block440, for each interval of the piece-wise mapping function, set theendpoints to be the threshold points. Refer to FIG. 6, which is anillustration of using threshold points and true dynamic range to segmentthe color components and calculate color transforms for differentdevices. The Device A color component (in this example, 330-1) has atheoretical dynamic range 605 from zero to a theoretical maximum (e.g.,zero to 255). The interval 1 (one) is bounded by zero and the thresholdpoint #k. The interval 2 (two) is bounded by the threshold point #k andthe true upper bound, which is described in more detail below in anexemplary embodiment. In other exemplary embodiments, it is noted thatthe true upper bound can be set to a predetermined value or set to thetheoretical maximum. A sub-transform 1 (one) is applied to the interval1 (one) to map to an interval 3 (three) on Device B (e.g., a referencedevice). Similarly, sub-transform 2 (two) is applied to the interval 2(two) to map to an interval 4 (four) on Device B (e.g., a referencedevice). The color component 330-1 has an actual range 610 from zero toa true upper bound and this actual range 610 is mapped to the actualrange 620 of the color component 360-1 (in this example), using twosub-transforms (in this example). Reference 615 illustrates exemplaryoperations of a transform 315.

The threshold points (e.g., threshold point #k and the true upper bound)are used to segment the full dynamic range of a color component intosegments, so that in each segment, the histogram of Device A willapproximate the reference device. The positions of threshold points areusually placed around locations where there are the largest histogramdistortions. More particularly, the threshold points are estimated sothat the piece-wise transforms will optimally compensate for thehistogram differences. That is, one estimates the threshold points sothat the distortion of color in each range of the histogram will becompensated. Typically, one chooses two or three locations to segmentthe full range of the histogram, depending on how the color distortionsare distributed.

Turning to FIG. 7, this figure illustrates a pipeline for determiningtrue dynamic range of an image relative to a mobile device. In block710, the reference dynamic range of pixel intensity levels is defined.These levels include upper range and lower range level values. Usuallythe range is determined by two different ways: either by setting therange the same as the range from the reference device, or setting therange as the biggest dynamic range possible on Device A (for example,eight bits per channel means 0-255).

In block 720, the pixel values of the image are sorted. In block 730,the highest M and lowest M batch of intensity values are selected. Atypical, non-limiting, value for M is one-fifth of the total number ofpixels in the image. In block 740, the true upper and lower range valuesof the image are calculated by taking the average intensity of thehighest M and lowest M batches of pixel intensities. This process avoidsthe influence of image noise on the true dynamic range. The output isthe true dynamic range 750 of Device A. It is noted that in an exemplaryembodiment, the true dynamic range 750 is the same as the actual range610 shown in FIG. 6. That is, the true upper bound of FIG. 6 is set asthe largest value (the “upper bound”) for the true dynamic range 750. Itis further noted that FIG. 6 assumes the range for the exemplary colorcomponent 330-1 starts at zero. However, the true dynamic range 750 mayalso have a non-zero lower bound (shown in FIG. 7 as “lower bound”) andthe value for the lower bound may be used in FIG. 6 instead of zero.

In block 760, the mapping of true dynamic range 750 to the referencedynamic range (e.g. actual range 620 in FIG. 6) is determined, using apiece-wise transform (e.g., and taking into account the white balancefactors described above in reference to FIG. 4). The factors include thethreshold points that decide the intervals and a linear factor thateither compresses or stretches a corresponding interval to match DeviceA to the reference device. It is noted that it is also possible to havelinear factor of one, which leave the range of interval unchanged, buttypically the end points of the interval will be moved within the fullrange of the histogram. The process uses a true dynamic range 750 thatcan be applied to the color components of Device A, e.g., through atransform 315 and/or as a true upper bound as shown in FIG. 6. Forinstance, the true dynamic range 750 can be used to stretch or compress(or leave the same) the actual range 610 of the color component to matchthe actual range 620 of the color component for the reference device(Device B in FIG. 6).

Without in any way limiting the scope, interpretation, or application ofthe claims appearing below, a technical effect of one or more of theexample embodiments disclosed herein includes mapping image data fromone color space defined by one device into a color space defined byanother device. Another technical effect of one or more of the exampleembodiments disclosed herein is adjusting one or more colorcharacteristics of one or more color components to be closer tocorresponding one or more color characteristics of a reference device,the modifying creating a modified image.

Embodiments of the present invention may be implemented in software,hardware, application logic or a combination of software, hardware andapplication logic. In an example embodiment, the application logic,software or an instruction set is maintained on any one of variousconventional computer-readable media. In the context of this document, a“computer-readable medium” may be any media or means that can contain,store, communicate, propagate or transport the instructions for use byor in connection with an instruction execution system, apparatus, ordevice, such as a computer, with one example of a computer described anddepicted in FIG. 1. A computer-readable medium may comprise acomputer-readable storage medium that may be any media or means that cancontain or store the computer program code for use by or in connectionwith an instruction execution system, apparatus, or device, such as acomputer.

It is noted that the embodiments may also be performed by means. Forinstance, an apparatus could comprises a means for modifying values ofimage data of a first image of an object, the first image taken by auser using the apparatus, the modifying performed to map at least onecolor characteristic of at least one color component of the apparatus toa corresponding at least one color characteristic for the at least onecolor component of a reference device, the modifying creating a modifiedimage; and a means for performing comparisons between the modified imageand a plurality of second images of objects taken by the referencedevice.

Additionally, the embodiments may be implemented in a computer program.More specifically, a computer program may comprise code for code formodifying values of image data of a first image of an object, the firstimage taken by a user using a user equipment, the modifying performed tomap at least one color characteristic of at least one color component ofthe user equipment to a corresponding at least one color characteristicfor the at least one color component of a reference device, themodifying creating a modified image; and code for performing comparisonsbetween the modified image and a plurality of second images of objectstaken by the reference device, when the computer program is run on aprocessor.

In another exemplary embodiment, the computer program according to theprevious paragraph is a computer program product comprising acomputer-readable medium bearing computer program code embodied thereinfor use with the user equipment.

In a further exemplary embodiment, an apparatus includes at least oneprocessor; and at least one memory including computer program code. Theat least one memory and the computer program code configured to, withthe at least one processor, cause the apparatus to perform at least thefollowing: modifying values of image data of a first image of an object,the first image taken by a user using the apparatus, the modifyingperformed to map at least one color characteristic of at least one colorcomponent of the apparatus to a corresponding at least one colorcharacteristic for the at least one color component of a referencedevice, the modifying creating a modified image; and performingcomparisons between the modified image and a plurality of second imagesof objects taken by the reference device.

If desired, the different functions discussed herein may be performed ina different order and/or concurrently with each other. Furthermore, ifdesired, one or more of the above-described functions may be optional ormay be combined.

Although various aspects of the invention are set out in the independentclaims, other aspects of the invention comprise other combinations offeatures from the described embodiments and/or the dependent claims withthe features of the independent claims, and not solely the combinationsexplicitly set out in the claims.

It is also noted herein that while the above describes exampleembodiments of the invention, these descriptions should not be viewed ina limiting sense. Rather, there are several variations and modificationswhich may be made without departing from the scope of the presentinvention as defined in the appended claims.

1. A method, comprising: modifying values of image data of a first imageof an object using a processor, the first image taken by a user using auser equipment, the modifying performed to map at least one colorcharacteristic of at least one color component of the user equipment toa corresponding at least one color characteristic for the at least onecolor component of a reference device, the modifying creating a modifiedimage; and on the user equipment, performing comparisons between themodified image and a plurality of second images of objects taken by thereference device.
 2. The method of claim 1, wherein performingcomparisons further comprises: selecting an image for presentation tothe user based on the comparisons; and allowing the user to confirm ifthe presented image is an image of the object in the first image.
 3. Themethod of claim 2, further comprising, in response to the userconfirming the presented image is an image of the object in the firstimage, presenting information related to the object to the user.
 4. Themethod of claim 2: further comprising, in response to the userconfirming that the presented image is an image of the object in thefirst image, creating a transformation that maps the at least one colorcharacteristic of the at least one color component of the first image toa corresponding the at least one color characteristic for the at leastone color component of the reference device; and wherein modifyingvalues further comprises applying the transformation to the values ofthe image data corresponding to the at least one color component.
 5. Themethod of claim 1, wherein modifying further comprises applying amapping to the values, the mapping converting white balance of the atleast one color component of the user equipment to a corresponding whitebalance for the at least one color component of the reference device. 6.The method of claim 5, wherein the mapping is a first mapping andmodifying further comprises applying a second mapping to the values, thesecond mapping converting contrast, saturation, or both contrast andsaturation of the at least one color component of the user equipment toa corresponding contrast, saturation, or both contrast and saturationfor the at least one color component of the reference device.
 7. Themethod of claim 1, wherein each value of image data has a componentcorresponding to each of the at least one color components, and whereinthere are three color components: one color component for red, one colorcomponent for blue, and one color component for green.
 8. The method ofclaim 1: further comprising creating a transformation that maps the atleast one color characteristic of the at least one color component ofthe user equipment to a corresponding at least one color characteristicfor the at least one color component of the reference device; andwherein modifying values further comprises applying the transformationto the values of the image data corresponding to the at least one colorcomponent.
 9. The method of claim 8, wherein creating a transformationfurther comprises for each of the at least one color components:computing a histogram for the color component on the user equipment andon the reference device; transforming the color component of the userequipment if histogram distortion exceeds a preset threshold;determining a piece-wise linear mapping of the color component for theuser equipment being transformed, the piece-wise linear mappingconverting color component values of the user equipment to colorcomponent values of the reference device; and for each interval of thepiece-wise mapping function, setting endpoint of the interval to be thethreshold points.
 10. The method of claim 9, wherein creating atransformation further comprises, for each of the at least one colorcomponents on the user equipment: defining a reference dynamic range ofpixel intensity values; sorting pixel intensity values in the firstimage; selecting the highest M values and the lowest M values; averagingthe highest M values; averaging the lowest M values; determining a truedynamic range based on the averages; calculating, using a piece-wisetransform, a mapping of the true dynamic range to a reference dynamicrange of the reference device.
 11. The method of claim 10, wherein eachof the true dynamic ranges is used for corresponding at least one colorcomponents when transforming the corresponding at least one colorcomponents of the user equipment if histogram distortion exceeds apreset threshold.
 12. An apparatus comprising: at least one processorconfigured to cause the apparatus to perform at least the following:modifying values of image data of a first image of an object, the firstimage taken by a user using the apparatus, the modifying performed tomap at least one color characteristic of at least one color component ofthe apparatus to a corresponding at least one color characteristic forthe at least one color component of a reference device, the modifyingcreating a modified image; and performing comparisons between themodified image and a plurality of second images of objects taken by thereference device.
 13. The apparatus of claim 12, wherein performingcomparisons further comprises: selecting an image for presentation tothe user based on the comparisons; and allowing the user to confirm ifthe presented image is an image of the object in the first image. 14.The apparatus of claim 13: wherein the at least one processor is furtherconfigured to cause the apparatus to perform, in response to the userconfirming that the presented image is an image of the object in thefirst image, creating a transformation that maps the at least one colorcharacteristic of the at least one color component of the first image toa corresponding the at least one color characteristic for the at leastone color component of the reference device; and wherein modifyingvalues further comprises applying the transformation to the values ofthe image data corresponding to the at least one color component. 15.The apparatus of claim 12, wherein modifying further comprises applyinga mapping to the values, the mapping converting white balance of the atleast one color component of the apparatus to a corresponding whitebalance for the at least one color component of the reference device.16. The apparatus of claim 15, wherein the mapping is a first mappingand modifying further comprises applying a second mapping to the values,the second mapping converting contrast, saturation, or both contrast andsaturation of the at least one color component of the apparatus to acorresponding contrast, saturation, or both contrast and saturation forthe at least one color component of the reference device.
 17. Theapparatus of claim 12, wherein each value of image data has a componentcorresponding to each of the at least one color components, and whereinthere are three color components: one color component for red, one colorcomponent for blue, and one color component for green.
 18. The apparatusof claim 12: wherein the at least one processor is further configured tocause the apparatus to perform creating a transformation that maps theat least one color characteristic of the at least one color component ofthe apparatus to a corresponding at least one color characteristic forthe at least one color component of the reference device; and whereinmodifying values further comprises applying the transformation to thevalues of the image data corresponding to the at least one colorcomponent.
 19. The apparatus of claim 12, further comprising at leastone memory comprising computer program code, the at least one memory andthe computer program code configured to, with the at least oneprocessor, cause the apparatus to perform the modifying the values andthe performing the comparisons.
 20. A computer program productcomprising a computer-readable storage medium bearing computer programcode embodied therein fox use with, an apparatus, the computer programcode comprising: code for modifying values of image data of a firstimage of an object, the first image taken by a user using the apparatus,the modifying performed to map at least one color characteristic of atleast one color component of the apparatus to a corresponding at leastone color characteristic for the at least one color component of areference device, the modifying creating a modified image; and code forperforming comparisons between the modified image and a plurality ofsecond images of objects taken by the reference device.